
## Splitgraph helps teams collaborate on data

At Splitgraph, we're convinced there's better ways to collaborate on data than
sharing CSV files. The ability to
[query other databases](/blog/postgresql-fdw-aggregation-pushdown-multicorn-part-1)
using Postgres queries, import data from hundreds of data sources,
[transform it with dbt](/blog/airbyte-dbt-splitgraph),
and travel back in time to any previous version all serve the same
purpose: to give teams working with data a place to collaborate effectively.
That said, if you're working with CSV files, [we have that covered as well 😉](/blog/csv-to-sql-upload-to-data-delivery-network-ddn).

The quickest way to try Splitgraph is to simply sign up to Splitgraph.com, the
public data portal open to everyone. You can import or connect your own data
into private or public repositories. And you can query over 40,000 public data
tables within 60 seconds, just by
[pointing your Postgres client to `data.splitgraph.com:5432`](https://www.splitgraph.com/connect/query).

## Dedicated Data Portals

For organizations, we also provide **Dedicated Data Portals**, which are private
deployments of Splitgraph to the cloud infrastructure of your choice. You get
all the same features of Splitgraph.com, except within a dedicated single-tenant
instance that you can invite your teammates to.

Dedicated Data Portals are currently in beta. Starting today, you can provision
a demo environment free for 7 days with no credit card required. If you want to
continue experimenting, just get in touch with us and we'll extend the trial.
Once you're ready to buy, schedule a meeting to discuss your requirements and pricing.

## Try Splitgraph in a single-tenant environment

In addition to the features available on the public [Splitgraph.com](https://www.splitgraph.com),
a Dedicated Data Portal enables you to:

- Store your data in a single-tenant environment.
- Get your own Splitgraph subdomain.
- Invite colleagues, collaborators and customers.
- Run Splitgraph in your cloud provider's region to minimize latency and transfer costs.
- Integrate with your company's OAuth/SSO (e.g. Okta).
- Gain in-depth control over sharing and unlimited namespaces.

## Deploy now

Launch your own demo Dedicated Data Portal with just a few
clicks (signup, but no credit card required).

<ButtonLink href="/demo">Deploy Demo</ButtonLink>

## What to expect

After you signup and deploy the demo, we'll send you an email in about 20 minutes,
once your deployment is ready. It will look like this:

<img src="/static-v2/images/blog/demo/demo-welcome-email.png" />

Just click the button in the email to get started with your Dedicated Data Portal.


Deploy a demo instance of a Dedicated Data Portal, so you can experiment with Splitgraph features in a single-tenant environment. Try it today, free for 7 days, no credit card required.

Get your own private Splitgraph data portal


## What is Splitgraph?

Splitgraph is building the Unified Data Stack – an integrated and modern solution
for working with data without worrying about its infrastructure.

You can try Splitgraph now! Browse
[the public data catalog](https://www.splitgraph.com/explore) to discover
40,000+ repositories to query with SQL, through the
[web IDE](https://www.splitgraph.com/query)
or [any Postgres client](https://www.splitgraph.com/connect).

Splitgraph is powered by open standards and simple abstractions, like
[data images](https://www.splitgraph.com/docs/concepts/images) – immutable tables
that you can push and pull, or query on the fly.

We're not replacing SQL as our favorite query language anytime soon 😄, but we rely on [GraphQL](https://graphql.org/) to implement many of Splitgraph's internal APIs. Thanks to GraphQL schema stitching, we were able to add new backend services which extend the set of fields published by older services. Clients don't need to specify which service provides which field, this is determined automatically by the schema stitching logic.

## How we use GraphQL at Splitgraph

Projects like [Apollo Client](https://www.apollographql.com/docs/react/), [graphql-playground](https://github.com/graphql/graphql-playground) and [PostGraphile](https://www.graphile.org/postgraphile/) have made working with GraphQL a smooth experience, but the declarative query language has proven to be even more important than the tooling. It's given us the flexibility to develop services which can be used independently, but also as components implementing the following unified API:

```graphql
type Query {
  """Returns a single `Repository` identified
     by it's `namespace` and `repositoryName`."""
  metadataRepository(
    namespace: String!,
    repositoryName: String!): Repository
  authorizationRepository(
    namespace: String!,
    repositoryName: String!): Repository
  """Returns a single `User` based on their username."""
  authorizationUser(username: String!): User
  metadataUser(username: String!): User
}

type Repository {
  namespace: String!,
  repositoryName: String!,
  description: String,
  url: String,
  """Lists all users who have permission to read this repository."""
  usersWithReadAccess: [User!]!
}

type User {
  username: String!,
  fullName: String,
  """Lists all repositories this user has permission to read."""
  repositories: [Repository!]!
}
```

The "dashboard" page which greets users upon login is one of the many clients of this API. It needs to display the current user's name as well a link to each repository they have access to. A single GraphQL query can fetch the required fields:

```graphql
query Dashboard($username: String!) {
  metadataUser(username: $username) {
    fullName,
    repositories {
      description,
      url
    }
  }
}
```

## Schema stitching in the gateway service

Two separate services store the data requested by the `Dashboard` query. The client sends the query to the `gateway` service, which uses [schema stitching](https://www.graphql-tools.com/docs/schema-stitching/stitch-combining-schemas#combining-schemas) to combine the APIs of the backing `metadata` and `authorization` services into the unified schema shown above. As the `gateway` service executes the `Dashboard` query, it queries the backing services.

### Metadata service

The `metadata` service is powered by [PostGraphile](https://www.graphile.org/postgraphile/) and predates the `authorization` service. Before developing the latter, Splitgraph's internal GraphQL API Schema looked similar to the following:

```graphql
type Repository {
  namespace: String!
  repositoryName: String!
  description: String
  url: String
}

type User {
  username: String!
  fullName: String
}

type Query {
  metadataRepository(
    namespace: String!,
    repositoryName: String!): Repository
  metadataUser(username: String!): User
}
```

### Authorization service

We built the `authorization` service based on [Ory Keto](https://github.com/ory/keto) to keep track of user-repository permissions.
The `authorization` service's schema extends the types introduced by the `metadata` service:

```graphql
"""This Repository type is merged with the Repository
   type declared in the metadata service schema."""
type Repository {
  """namespace and repositoryName are the key fields
     by which the repository types in the two
     subschemas can be merged."""
  namespace: String!
  repositoryName: String!
  usersWithReadAccess: [User!]!
}

"""Similar to Repository, User will be merged
   with the metadata service's User type."""
type User {
  """username is the key field of User."""
  username: String!
  repositories: [Repository!]!
}

type Query {
  """authorization service specific fields for retrieving
     User and Repository objects."""
  authorizationRepository(
    namespace: String!,
    repositoryName: String!): Repository
  authorizationUser(username: String!): User
}
```

## What is schema stitching?

The main idea is simple: the different fields of a type may be distributed among several services as long as the corresponding subschemas all use the same type name. By default, the "stitched" schema just combines all the fields of any given type from all subschemas, but accidental type merging [can be avoided](https://www.graphql-tools.com/docs/schema-stitching/stitch-combining-schemas#duplicate-types).
Single-service GraphQL schemas operate under the closed world assumption that all fields belonging to a type are declared in the schema. Stitching leads to an open world model where any type may be extended with additional fields declared in a new subschema.

At first, the doubled fields in the `Query` type may seem superfluous. They have the same return types after all: `User` and `Repository`.
Why they're necessary becomes apparent as soon as we examine how the `Dashboard` query is processed by the `gateway` service.

## Querying stitched schemas

The `gateway` service uses the [merging flow](https://www.graphql-tools.com/docs/schema-stitching/stitch-type-merging#merging-flow) to execute queries and delegate to the other services.
With the proper [subschema configuration](https://www.graphql-tools.com/docs/schema-stitching/stitch-combining-schemas#subschema-configs), the schema stitching code can determine which service should be queried for a particular field.

Consider the steps required to compute results for the `Dashboard` query for user `mrDorp`.
1. The client submits the `Dashboard` query to the gateway service, which selects the `fullName` and `repositories` fields on the result of `metadataUser`.
    <img src="/static-v2/og/blog/dashboard-query-1.svg" />

1. Since `metadataUser` comes from the `metadata` service's subschema, the `gateway` queries it for the `fullName` field.
    <img src="/static-v2/og/blog/dashboard-query-2.svg" />

    ```graphql
    metadataUser(username: "mrDorp") {
      __typename
      fullName
      username
    }
    ```
    The `username` field is added implicitly since it's the key field used to join the `User` types in the two services. `__typename` is always added by the schema stitching code. The response to the query is:
    ```json
    {
      "data": {
        "metadataUser": {
          "__typename": "User",
          "username": "mrDorp",
          "fullName": "Ralph Dorp"
      }
    }
    ```

1. The `repositories` field of the `User` type belongs to the `authorization` service, so the `gateway` queries it next. The subschema configuration specifies that the top-level `authorizationUser` field may be queried to get the `User` fields declared in the service's subschema.
    <img src="/static-v2/og/blog/dashboard-query-3.svg" />

    ```graphql
    authorizationUser(username: "mrDorp") {
      username
      repositories {
        __typename
        namespace
        repositoryName
      }
    }
    ```
    Note that the `Dashboard` query selected the `description` and `url` fields of each `Respository` object, but these fields are declared by the `metadata` service schema. In the request headed for the `authorization` service, all that can be queried are the key fields. Just as in the previous query, the `gateway` service implicitly adds the `username` key field to the selection set. The response is the following:

    ```json
    {
      "data": {
        "authorizationUser": {
          "__typename": "User",
          "username": "mrDorp",
          "repositories": [
            {
              "__typename": "Repository",
              "namespace": "austintexas-gov",
              "repositoryName": "austin-high-school-graduation-rates-xeb7-q8v3"
            },
            {
              "__typename": "Repository",
              "namespace": "bts-gov",
              "repositoryName": "county-transportation-profiles-qdmf-cxm3"
            }
          ]
      }
    }
    ```
1. Having obtained the repositories' key fields, the `gateway` may query additional `Repository` fields from the `metadata` service. It consults the stitching configuration for the top-level field to query - in this case `metadataRepository`.
    <img src="/static-v2/og/blog/dashboard-query-4.svg" />

    ```graphql
    metadataRespository(
      namespace: "austintexas-gov",
      repositoryName: "austin-high-school-graduation-rates-xeb7-q8v3") {
        __typename
        namespace
        repositoryName
        description
        url
    }

    metadataRespository(
      namespace: "bts-gov",
      repositoryName: "county-transportation-profiles-qdmf-cxm3") {
        __typename
        namespace
        repositoryName
        description
        url
    }
    ```

    Just as with the query for the `authorizationUser` field, the key fields (`namespace` and `repositoryName`) and `__typename` are implicitly added to the query to allow merging of result objects.
    We created a PostGraphile plugin using GraphQL's [DataLoader](https://www.graphql-tools.com/docs/connectors#dataloader-and-caching) to combine the two queries above into a single GraphQL query ([drop us a line](mailto:support@splitgraph.com) if you're interested in using it). Unfortunately, the current schema does not allow for such optimization, so each repository's metadata fields are queried separately. The `metadata` service responds with:
    ```json
    {
      "data": {
        "metadataRepository": {
          "__typename": "Repository",
          "namespace": "austintexas-gov",
          "repositoryName": "austin-high-school-graduation-rates-xeb7-q8v3"
          "description": "Graduation rates for Austin high schools for years 2012 to 2016 provided by the Texas Education Agency.",
          "url": "https://www.splitgraph.com/austintexas-gov/austin-high-school-graduation-rates-xeb7-q8v3"
        }
      }
    }

    {
      "data": {
        "metadataRepository": {
          "__typename": "Repository",
          "namespace": "bts-gov",
          "repositoryName": "county-transportation-profiles-qdmf-cxm3"
          "description": "Profiles of transportation features of U.S. counties",
          "url": "https://www.splitgraph.com/bts-gov/county-transportation-profiles-qdmf-cxm3"
        }
      }
    }
    ```

1. The `gateway` has all the fields required to respond to the `Dashboard` query. Key fields which weren't selected in the `Dashboard` query are discarded once the objects have been merged.
    ```json
    {
      "data": {
        "metadataUser": {
          "__typename": "User",
          "fullName": "Ralph Dorp",
          "repositories": [
            {
              "__typename": "Repository",
              "description": "Graduation rates for Austin high schools for years 2012 to 2016 provided by the Texas Education Agency.",
              "url": "https://www.splitgraph.com/austintexas-gov/austin-high-school-graduation-rates-xeb7-q8v3"
            },
            {
              "__typename": "Repository",
              "description": "Profiles of transportation features of U.S. counties",
              "url": "https://www.splitgraph.com/bts-gov/county-transportation-profiles-qdmf-cxm3"
            }
          ]
        }
      }
    }
    ```

## Configuring subschemas to stitch

While processing the `Dashboard` query, the `gateway` had to decide which service to query for each requested field. This is determined by the [subschema configuration](https://www.graphql-tools.com/docs/schema-stitching/stitch-combining-schemas#subschema-configs).
Below is the simplified code for the `gateway` service, which is based on the TypeScript examples for [remote schemas](https://www.graphql-tools.com/docs/remote-schemas#graphql-over-http-executor-with-cross-undici-fetch) and [schema stitching](https://www.graphql-tools.com/docs/schema-stitching/stitch-combining-schemas#getting-started):

```typescript
import { stitchSchemas } from "@graphql-tools/stitch";
import { fetch } from "cross-fetch";
import { print, GraphQLSchema } from "graphql";
import { introspectSchema, wrapSchema } from "@graphql-tools/wrap";
import type { AsyncExecutor } from "@graphql-tools/utils/executor";
import { SubschemaConfig } from "@graphql-tools/delegate";

const authorizationServiceUrl = "http://api.splitgrph.com/authorzation/graphql";
const metadataServiceUrl = "http://api.splitgrph.com/metadata/graphql";

const makeRemoteExecutor: (serviceUrl: string) => AsyncExecutor = (
  serviceUrl: string
) => async ({ document, variables }) => {
  const query = print(document);
  const result = await fetch(serviceUrl, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ query, variables }),
  });
  return result.json();
};

const defineSubSchema = async (
  serviceUrl: string,
  merge?: SubschemaConfig["merge"]
): Promise<SubschemaConfig> => {
  const executor = makeRemoteExecutor(serviceUrl);
  const schema = wrapSchema({
    schema: await introspectSchema(executor),
    executor,
  });
  return { executor, schema, merge };
};

export const makePublicSchema = async (): Promise<GraphQLSchema> =>
  stitchSchemas({
    subschemas: [
      await defineSubSchema(authorizationServiceUrl, {
        User: {
          fieldName: "authorizationUser",
          selectionSet: "{ username }",
          args: ({ username }) => ({ username }),
        },
        Repository: {
          fieldName: "authorizationRepository",
          selectionSet: "{ namespace repositoryName }",
          args: ({ namespace, repositoryName }) => ({
            namespace,
            repositoryName,
          }),
        },
      }),
      await defineSubSchema(metadataServiceUrl, {
        User: {
          fieldName: "metadataUser",
          selectionSet: "{ username }",
          args: ({ username }) => ({ username }),
        },
        Repository: {
          fieldName: "metadataRepository",
          selectionSet: "{ namespace repositoryName }",
          args: ({ namespace, repositoryName }) => ({
            namespace,
            repositoryName,
          }),
        },
      }),
    ],
  });
```

The `GraphQLSchema` object returned by `makePublicSchema` can be used to create an [HTTP GraphQL endpoint](https://www.graphql-tools.com/docs/server-setup#setup-an-http-server).

In order to query a subschema, it's URL must be known. Conveniently, a GraphQL query can be used to obtain an endpoint's schema. This is what `defineSubSchema()` does when it calls `introspectSchema()`. The object passed to `defineSubSchema()` is the [merge configuration](https://www.graphql-tools.com/docs/schema-stitching/stitch-type-merging#type-merging). Consider the fragment,

```javascript
  await defineSubSchema(authorizationServiceUrl, {
    User: {
      fieldName: "authorizationUser",
      selectionSet: "{ username }",
      args: ({ username }) => ({ username }),
    }
```

It declares: when fields of the `User` type are queried which were declared by the `authorization` service schema, they can be obtained by passing the `username` field of the pre-existing `User` object as the argument to the top-level `authorizationUser` field.

This was employed in step 2 of the query process described earlier, when the `User.repository` field was merged with the `User` object obtained from the `metadata` service in the previous step.

`fieldName` specifies the field of the top-level `Query` type to consult. `selectionSet` defines the set of fields to be selected from the existing `User` object to get `authorizationUser`'s arguments, typically the key field or fields. `selectionSet` could contain fields not yet available on the current `User` instance. In such a case, an additional subschema query would be made before the field referred to by `fieldName` is queried.

The `args` function transforms the existing fields on the `User` object to be used as arguments to `authorizationUser`. In most cases, this is the identity function, but it can be useful for things like string to number conversion, especially when stitching schemas for APIs outside of our control. For example, one could extend the types in the official Shopify GraphQL Schema with custom fields routed to an internal service.

It would also be possible to start by querying the `authorization` service:

```graphql
query Dashboard2($username: String!) {
  authorizationUser(username: $username) {
    fullName,
    repositories {
      description,
      url
    }
  }
}
```

The steps to executing the `Dashboard2` query would be:
1. Query the `authorization` service for the `username` key field and the `repositories` field.
1. Merge the `User.fullName` field to the existing `User` object instance by querying `metadataUser`.
1. Merge `url` and `description` fields with the existing `Repository` objects using `metadataRepository`.

## Conclusion

Schema stitching has enabled us to extend an existing service's GraphQL API with new fields served by a new service. The "stitching" is seamless in the sense that clients don't need to know which field belongs to which service's subschema. We evolved our API without affecting earlier queries.


Read about we use GraphQL schema stitching to provide a single coherent schema for accessing several services with shared types from overlapping GraphQL schemas.

Splitgraph has been acquired by EDB! Read the blog post.

By Peter Neumark

Get your own private Splitgraph data portal

Combining multiple GraphQL backends with schema stitching

Product

Support

Company

Splitgraph

Splitgraph has been acquired by EDB! Read the blog post.

By Peter Neumark

Get your own private Splitgraph data portal

Combining multiple GraphQL backends with schema stitching

Product

Support

Company

Community

Splitgraph